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DETAILED ACTION 



Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1-5, 7, 10, 3 1, and 36-47 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Depta (US 6,549,122) in view of Ellis et al. (US 6,523,006). 

Re claims 1, 4, 5, 10, and 36, Depta teaches a system for aiding a person with a visual 
impairment or visual obstruction (fig. 2), comprising: 

means (1 1 of fig. 2) as at least on computerized imaging device for sensing time/space 
characteristics and physical characteristics information about a plurality of objects in a field of 
view said person (distance and locations of the objects are detected by the cameras 1 1 of fig. 2); 
means (13 and 14 of fig. 2) as Processing and Control Unit for converting said time/space 
characteristics and physical characteristics information (col. 4, lines 9-43) and for characterizing 
(recognition) the object (col. 3, line 58-col. 4, line 25, note converting the information taken by 
the cameras (1 1 of fig. 2) into a virtual three-dimensional image, and image detection unit for 
detecting or recognizing objects based on the information on the surroundings, which would 
suggest mean for identifying and interpreting the characteristics of the objects); and means as 
User Interface Module (e.g. 3 and 4 of fig. 2) for verbally for communicating output ((e.g. 15 
and 17 of fig. 2) and input information (Control Module inputs information to the Processing 
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Unit 14 of fig. 2) between said system and said user (col. 4, lines 3-15); wherein said means for 
interpreting said time/space and physical characteristics information comprises: a sensors 
processor (1 of fig. 2) and a control unit (4 of fig. 2) for interpreting the sensing data form the 
sensing means (26 and 1 1 of fig. 2) , and Vision Processing module (an object synthesis (13 of 
fig. 2) and recognition system (14 of fig. 2)) for interpreting the sensing data from said sensing 
means. 

It is noted that Depta does not particularly discloses the High-Level Vision processing 
module for interpreting high-level data from sensing means and a World Model database for 
performing said high-level characterizing as claimed. 

However, Ellis teaches the High-Level Vision processing module (1 10 of figs. 3 A and 
3B; col. 3, lines 36-56) for interpreting, decomposing, and synthesizing (220 of fig. 2) high-level 
data from sensing means (A and B of figs. 3A and 3B) and a World Model database (330 of fig. 
3B) for performing said high-level characterizing. 

Therefore, taking the teachings of Depta and Ellis as a whole, it would have been obvious 
to one of ordinary skill in the art to modify the teachings of Ellis into the system of Depta to 
provide a specific description of the three-dimensional object to guide a visual or vision impaired 
person precisely. 

Re claims 2 and 37, Depta further teaches wherein said means for sensing further 
comprises devices selected from the group consisting of: CCD sensors (1 1 of fig. 2, Note 
cameras 1 1 inherently have CCD sensors). 

Re claim 3, Depta further teaches wherein said time/space characteristics and physical 
characteristics information about an object includes any of: physical dimensions, general shape 
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description, texture, color, the distance and position of said user from said object, motion of said 
object, spatial relationships between objects (col. 3, line 58-col. 4, lines 3; Note three 
dimensional image). 

Re claim 31, Depta further teaches 3D object rendering (col. 3, lines 61 -col. 4, line 3). 

Re claim 38, the combination of Depta and Ellis further discloses wherein said World 
Model database (330 of fig. 3B, Ellis) comprises data selected from the group: physical 
descriptions of known objects and components thereof (col. 3, lines 24-36); the relations of said 
objects to their respective said components; common relations among said components; 
groupings of said known objects into classes and subclasses (chairs, doors, tables arc classes and 
sub-classes); common properties of said classes, subclasses and groupings (shapes and sizes of 
chairs, tables, doors); and common relations among and between said classes, subclasses and 
groupings. 

Re claims 7 and 39-42, the combination of Depta and Ellis further teaches a World Model 
database (330 of fig. 3B; Ellis) is continuously enriched by adoption to new experiences of said 
person and implemented in at least one option selected from the modes: direct teaching mode; 
generalization teaching mode by positive enforcement or default; and refinement teaching mode 
by negative enforcement or correction (col. 3, liens 26-32; Ellis); wherein said direct teaching 
mode comprises giving a name to a part of a scene by said user (col. 3, lines 33-34); wherein 
said generalization teaching mode comprises identifying a part of a given scene as belonging to a 
specific object class, although some of the relationships, or some of the components, may deviate 
from that identified in said database (col. 3, lines 37-56); wherein said refinement teaching mode 
comprises refining the set of definitions to permanently exclude in the future an identification of 
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only part of a scene as belonging to a specific object class when said user has rejected said partial 
identification (new items are added to library and verbal descriptions to new or rare objects as 
indicated, see col. 3, lines 36-36). 

Re claim 43, Depta further teaches wherein said means for interpretation comprises scene 
interpretation data and sensed data streams (1 1 of fig. 2). 

Re claim 44, Depta further teaches said means for interpretation (13 and 14 of fig. 2) 
further comprises color information of said plurality of objects (col. 3, lines 54-55) and Ellis 
further teaches wherein said means for interpretation further comprises computations for relating 
distances (col. 3, lines 3- 1 8) and components thereof within the field of vision of said person 
(col. 2, lines 63-65). 

Re claim 45, Depta further teaches wherein said output information comprises said 
characterization and interpretation of said plurality of objects (e.g. 13 of fig. 2, note image 
detection unit for detecting or recognizing objects based on the information on the surroundings). 

Re claim 46, the combination of Depta and Ellis further teaches wherein said output 
information further comprises a scene description about said surrounding environment within the 
field of view of said person (Ellis; note verbal descriptions of new or rare objects within the field 
of view is considered a scene description, new objects are in the scene, col. 3, lines 32-36; lines 
47-56). 

Re claim 47, the combination of Depta and Ellis further teaches wherein said scene 
description is verbally communicated in conversational speech to said person based on the 
distance of said plurality of objects to said user, the closest being communicated first as having a 
higher priority (Ellis; col. 3, lines 47-56). 
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3. Claims 27, 31, 35, 52, and 53 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ellis et al. (US 6,523,006) in view of Sussman (US 6,198,395). 

Re claim 27, Ellis teaches a method for detecting, analyzing, identifying, and reporting a 
plurality of objects by name to a visually impaired or blind person in a field of view of said 
person (figs. 1 A and IB) comprising the steps of providing a 3-D imaging system to accurately 
measure both the coordinates of said plurality of objects in both static and dynamic motion and 
the characterization of the relationships between said plurality of objects in relation to said 
person (figs. 3A and 3B, col. 3, lines 7-36); acquiring a plurality of 3-D visual images and 
information about said plurality of objects (col. 3, lines 24-36); processing said plurality of 3-D 
visual images and said information (col. 3, lines 37-56); storing said information of said plurality 
of objects by names in a hierarchy utilizing a structured World Model (col. 3, lines 24-29; 330 of 
fig. 3B); identifying said plurality of objects, and communicating verbally with said person to 
report said information about said plurality of objects in said field of view (col. 3, lines 30-56). 

It is noted that Ellis does not particularly teach laser imaging device as claimed. 

However, Sussman teaches an array of laser sensors for receiving laser signals reflected 
from objects in the given areas with each laser sensor corresponding to a respective one of the 
laser transmitters (16 of fig. 1). 

Therefore, taking the teachings of Ellis and Sussman as a whole, it would have been 
obvious to one of ordinary skill in the art to use the laser sensors of Sussman into the method of 
Ellis in order to provide the blind with a means of traveling independently, confidently, and 
quickly. 
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Re claim 31, Ellis further teaches 3D object rendering (col. 2, lines 53-65; col. 5, lines 

6—64). 

Re claim 35, Ellis further teaches communicating the verbal description of the scene 
related to said plurality of objects in said field of vision to said person (col. 3, lines 30-36). 

Re claim 52, Ellis further teaches providing a process of learning new facts about 8aid 
physical surroundings of said person for growth and expansion of said World Model (col. 3, lines 
24-30). 

Re claim 53, Ellis further teaches wherein said World Model hierarchically represents 
existing knowledge of the physical surroundings of said person; classes of said plurality of 
objects; the common relationships among said classes; the relationships of said plurality of 
objects to their various components; and the way low-level components in said hierarchy can be 
composed o£ regular surfaces (col. 3, lines 37-64). 

4. Claims 26 and 50 are rejected under 35 U.S.C. 103(a) as being unpatentable over Depta 
(US 6,549,122) in view of Ellis et al. (US 6,523,006) as applied to claim 10, and further in view 
of Basson et al. (US 6,975,991). 

Re claims 26 and 50, the combination of Depta and Ellis teaches wherein said 
instructions comprise naming and classifying objects known to said user (col. 3, lines 24-20, new 
items is added and updated). 

It is noted that the combination of Depta and Ellis does not particularly teach an 
automatic speech recognition unit for verbal input of instructions from said person to said 
Processing and Control Unit. 
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However, Basson teaches an automatic speech recognition unit for verbal input of 
instructions from said person to said Processing and Control Unit (218 of fig. 2; Note the 
transcription service (218 of fig. 2), itself, may comprise: a human stenographer who transcribes 
the audio content, in real-time, into text; a voice e ( - N system which automatically 
recognizes the audio content, also in real-time, and outputs a textual representation of the 
decoded speech; or some combination of both human stenographer and automatic recognition; 
and the computer (200 of fig. 2) may execute a s;ve, v o 1 engine). 

Therefore, taking the teachings of Depta, Ellis, and Basson as a whole, it would have 
been obvious to one of ordinary skill in the art to use the speech recognition of Basson for 
carrying the combined apparatus of Depta and Ellis in order to allow the user to easily detect 
voice and moving around. 

Allowable Subject Matter 

5. Claims 48-49 and 5 1 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 

Conclusion 

6. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Contact Information 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tung Vo whose telephone number is 571-272-7340. The 
examiner can normally be reached on Monday-Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on 571-272-741 8. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
/Tung Vo/ 

Primary Examiner, Art Unit 2621 



